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ABSTRACT 

Efficient  algorithms  have  been  developed  recently  which  utilize  the 
specialized  structure  of  the  linear  programming  formulation  for  the  problem 
of  least  absolute  value  estimation.  These  algorithms  generally  proceed  In 
the  direction  of  steepest  descent  along  an  edge  of  a convex  polyhedral  sur- 
face. However,  we  will  show  that  the  extreme  point  path  of  steepest  descent 
may  not  be  taken  when  degeneracy  occurs.  We  will  also  present  a criterion 
that  determines  the  basic  edge  for  steepest  descent. 
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1.  Introduction 

Least  absolute  value,  or  Lj  norm,  estimates  have  long  been  considered 
as  an  acceptable  alternative  to  least  squares  estimates.  Fourier  (See  Darboux) 

[ appears  to  have  been  the  first  to  examine  the  problem  of  obtaining  such 

[ 

estimates.  Later,  Edgeworth  (1887,  1888,  1923)  also  investigated  the 
problem.  The  French  mathematician  de  la  Vallee  Poussin  (1911)  proposed 
a solution  procedure.  In  addition,  algorithms  were  presented  by  Rhodes  (1930) 
and  by  Singleton  (1940).  The  labor  involved  in  computing  the  least  absolute 
value  (LAV)  estimators  made  the  technique  unpopular,  and  for  a period  of 
time  most  results  were  restricted  to  the  cases  of  either  one  or  two  parameters. 
It  was  not  until  the  work  of  Charnes,  Cooper,  and  Ferguson  (1955)  that  a 
practical  procedure  for  obtaining  LAV  estimators  was  given.  They  demonstrated 
that  the  problem  of  minimizing  the  sum  of  absolute  deviations  could  be  reduced 
to  an  equivalent  linear  programming  (LP)  formulation. 

With  this  development  available  to  researchers,  there  has  been  a steadily 
increasing  interest  in  LAV  estimators  and  their  properties  in  recent  years. 

One  of  the  main  directions  of  this  research  was  to  find  efficient,  specialized 
algorithms  which  would  solve  the  linear  programming  formulation.  Of  particular 
interest  are  the  algorithms  of  Davies  (1967)  and  Barrodale  and  Roberts  (1973). 
These  closely  related  algorithms  use  the  special  structure  of  the  problem 
to  perform  "multiple  pivots"  before  actually  performing  an  iteration  of  the 
simplex  algorithm.  In  this  paper,  we  shall  point  out  that  these  multiple 
pivoting  techniques  will  not  always  give  the  extreme  point  path  of 
steepest  descent  when  degeneracy  occurs.  The  possibility  of  cycling  of 
bases  is  not  a problem  since  it  can  be  resolved  by  the  perturbation  technique 
of  Charnes  (1952). 
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2.  Linear  Programming  Formulation 

Mathematically,  the  least  absolute  value  problem  is  that  of  finding 
estimates  of  the  parameters  i » 1,  2,  ....  m which  solves  the  following 
problem. 


Minimi 


“ ^ •'‘Ji'i  •’‘jaV  •••  -Wi. 


where  (yj , •••>  J * It  2 n represents  the  observed  values 

during  n repetitions  of  an  experiment. 

The  linear  programming  equivalent  of  (1)  is  the  following. 


Minimi 


iraize  z (P.  + Nj 
j-1  ^ J 


subject  to  XjjBj  + XjgBg  + ...  +Xj,„B^  +Pj  -Nj  = ys  j = 1,  2,  . . . , n 

P..NjiO 

where  P^  and  Nj  are  the  positive  and  negative  deviations  of  the  j-th  obser- 
vation, repectively. 

Using  matrix  notation,  we  can  now  write  the  constraints  of  LP(2)  as 


follows: 


Xb  + IP  - IN  = Y 
P,  N > 0 


where  P^  - (Pj,  •••,  P„),  - (Nj,--*,  N„),  = (yj,'",  y„),  X is  an  n by 

m matrix  with  n > m,  and  B^  = (Bi,  * **,  B ). 

~ * m 
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3.  Multiple  Pivoting  Techniques  of  the  Special  Purpose  Algorithms 

Our  first  objective  in  this  section  is  to  present  a geometrical  inter- 
pretation of  the  L.  estimation  problem.  With  this  explanation,  the  steps 
^ n 

used  in  the  algorithms  will  be  more  evident.  Geometrically,  S = i:  ly^  - X^el 

i=l 

represents  a convex  ployhedral  surface  in  Euclidean  (m  + 1)  space  with  co- 
ordinates (8jt  ®ni’  where  = (x^l,  x^g*  '■'*  algorithm 

considers  basic  solutions  formed  by  m hyperplanes  X^g  = y^ , i = 1,  2,  • • • , m. 

If  8*  is  a solution  to  such  a system,  then  (8*,  S(8*))  is  an  extreme  point 
(vertex)  on  the  polyhedral  surface.  An  edge  on  the  polyhedral  surface  is 
determined  by  the  intersection  of  (m  - 1)  of  the  hyperplanes  X,-8  = > 

During  any  iteration,  the  special  purpose  algorithms,  use  m such  hyperplanes 
so  that  the  associated  X^  will  form  a basis  for  e"*.  The  intersection  of  these 

[ m hyperplanes  determine  the  current  basic  solution,  and  the  m edges  formed 

I 

I by  the  possible  combinations  of  these  m - 1 hyperplanes  are  called  the  basic 

edges.  The  algorithms  search  along  the  basic  edges  to  determine  if  a decrease 
in  the  sum  of  the  absolute  values  of  the  residuals  is  possible.  If  a decrease 
is  possible,  then  an  iteration  of  the  algorithm  will  move  along  the  chosen 
edge  in  the  prescribed  direction  by  pivoting  through  the  sequence  of  extreme 
points  on  this  special  path  until  a decrease  no  longer  occurs.  The  procedure 
is  repeated  until  the  lowest  vertex  is  reached  in  the  case  of  a unique  solution. 

We  shall  now  show  how  this  procedure  is  implemented  in  the  simplex  algorithm. 
The  standard  procedure  of  the  simplex  algorithm  is  to  choose,  first,  the  vector 
to  enter  the  basis  by  picking  the  most  negative  marginal  (reduced)  cost.  Then, 
the  vector  to  leave  the  basis  is  determined  by  the  minimum  ratio  test.  The 
pivoting  of  the  basis  vectors  then  follows.  However,  the  special  purpose 
algorithms  do  not  immediately  perform  the  pivot.  These  algorithms  add  twice 
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the  amount  of  the  component  of  the  current  tableau  associated  with  the 
minimum  ratio  to  the  reduced  cost.  If  this  new  reduced  cost  is  positive, 
then  the  pivot  operation  is  performed.  If  the  new  reduced  cost  is  negative, 
then  the  next  smallest  minimum  ratio  is  considered.  Twice  the  element  of 
the  tableau  associated  with  this  minimum  ratio  is  then  added  to  the  reduced 
cost.  The  procedure  continues  until  the  reduced  cost  becomes  nonnegative. 

At  that  time,  the  standard  pivot  procedure  is  implemented  on  this  last 
element  from  the  sequence  of  minimum  ratios.  At  each  of  these  "pivots",  a 
P^-  and  N^-  are  interchanged  in  the  basis  until  the  last  pivot  is  performed. 

In  order  to  formalize  this  mathematically,  we  shall  consider  LP(2). 

It  follows  directly  from  the  LP  theory  that  an  optimal  basis  must  contain 
all  m columns  of  the  matrix  X and  also  n - m columns  from  the  matrices  I 
and  -I  associated  with  the  2n  variables,  Pj  and  Nj.  We  should  note  that 
rank  deficiencies  are  easily  treated  within  the  LP  framework  by  the  techniques 
developed  for  LP  theory  in  Chames  and  Cooper  (1961).  Thus,  it  is  necessary 
to  consider  only  basis  matrices  of  the  form  (after  row  interchanges) 


where  E is  a matrix  with  +1  or  -1  on  the  diagonal  and  zeroes  off  the  diagonal. 
The  inverse  of  B is 


,-l  = 


EX„X‘^ 
R B 


The  reduced  cost  for  the  (n  - m)  nonbasic  N^{P^)  with  corresponding  P^(N^.) 
in  the  basis  is  +2.  For  the  case  where  both  and  P^,  m of  each,  are 
nonbasic,  the  reduced  cost  of  P.-  is  1 + e e-X.X-^.  x,  and  for  N.,  it  is 

1 -I^TP  J o\K)  1 

— 1 1 JEi** 

1 - E column  of  Xg  . IR  is  the  index 

set  of  X^'s  forming  X_,  and  e.  is  the  value  of  the  diagonal  element  of  E 

j K j 

associated  with  X..  The  optimality  conditions  are  then  -1  < E e.X.X  ,,  v l 1, 
J j^IR  J J 

k = 1,  2,  ...,  n. 

If  the  optimality  conditions  are  not  satisfied,  then  it  is  determined 
that  either  or  P^.  is  to  enter  the  basis,  and  the  variable  to  leave  the 
basis  is  determined  by  the  minimum  ratio  test.  It  is  given  as  follows: 


where  Yg  is  the  vector  of  the  dependent  variables  corresponding  to  Xg,  and 


f-1  if  P^ 
( 1 if 


enters  the  basis 
enters  the  basis. 
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In  order  to  illuminate  the  steps  of  the  multiple  pivots,  let  t be  the  value 

of  the  index  IR  which  yields  the  minimum  ratio.  Then  the  vector  to  be  removed 
from  the  basis  will  be  either  or  P^.  Suppose  that  it  is  and  that  P^. 
is  chosen  to  enter  the  basis.  If  1 + t ^ 

the  pivoting  procedure  is  performed.  Ifl+  z ^ 

jelR  ^ 

then  P^  will  replace  in  the  basis,  and  the  reduced  cost  must  be  updated. 

We  now  return  to  the  minimum  ratio  test  and  find  the  ratio  which  is  next 
to  the  smallest  in  the  index  set  IR.  The  procedure  is  continued  until  the 
reduced  cost  becomes  nonnegative,  and  then  an  iteration  is  carried  out. 
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4.  Criteria  for  Exploiting  Degeneracy 


Degeneracy  will  occur  if  there  are  more  than  m residuals  that  are  zero 

during  an  iteration.  We  shall  assume  that  there  are  (m+p)  residuals  that  are 

zero.  Geometrically,  this  will  occur  when  there  are  (m+p)  edges  through  a 

vertex  on  the  surface  of  the  convex  polytope  in  However,  if  we  consider 

fn  ^ 

E , then  B is  the  point  of  intersection  of  (m+p)  hyperplanes  X^B  = y^.  Hence 
the  2(m+p)  variables,  and  P^,  associated  with  these  equations  will  all  be 
zero,  and  thus  there  will  be  p basic  variables  which  are  zero.  Theoretically, 
these  p basic  variables  will  not  be  zero,  but  they  will  have  a "small"  value 
given  by  the  e-polynomial  when  the  perturbation  technique  of  Chames  (1952) 
is  applied.  The  perturbation  will  eliminate  the  possibility  of  cycling  of 
bases,  and  it  will  not  be  necessary  to  explicitly  determine  the  value  of  e. 

However,  these  algorithms  do  not  always  proceed  along  the  edge  of  steepest 
descent  when  degeneracy  occurs.  We  use  the  term,  edge  of  steepest  descent, 
to  refer  to  the  segment  of  an  edge  which  is  adjacent  to  a vertex  that  will  give 
the  greatest  rate  of  decrease  in  the  objective  function  when  moving  to  an  adja- 
cent vertex.  We  should  note  that  using  the  multiple  pivoting  technique  on  the 
edge  of  steepest  descent  may  not  give  the  specialized  extreme  point  path  that 
will  provide  the  greatest  decrease  in  the  objective  function.  There  may  be 
another  extreme  point  path  that  will  provide  a greater  decrease,  but  this 
cannot  be  determined  unless  iterations  of  the  algorithm  are  compared. 

We  shall  develop  our  results  on  the  assumption  that  it  is  possible  to 
obtain  a decrease  in  the  value  of  the  objective  function  along  one  of  the 
basic  edges.  However,  it  is  important  to  note  that  a decrease  may  not  be 
possible  along  any  of  the  m basic  edges.  Therefore,  it  is  possible  that 
several  degenerate  pivots  must  be  performed  in  order  to  obtain  a basis  which 
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has  an  edge  along  which  it  is  possible  to  obtain  a decrease.  This  situation 
may  cause  the  algorithm  to  cycle  if  a perturbation  technique  is  not  utilized. 

We  shall  now  illustrate  the  method  to  btain  the  edge  of  steepest  descent 
from  the  basic  edges.  Using  matrix  notation,  we  can  represent  the  constraints 
of  the  linear  programming  formulation  as  follows: 


After  multiplying  by  Xg^,  we  obtain  the  following 

« " - "iX  = *6*  ^B 

-EXrX-jVj  * EPr  4 EXrX;\  -ENr  . E(yR  - XrX^Vr). 


equations: 


If  the  solution  is  not  degenerate,  then  these  equations  will  simplify  as  below: 


e = *6*  J'B 

° ^^^R  ■ *R*b  ^B* 


where  either  > 0 or  > 0 and  = 0 for  all  i e IR.  If  degeneracy 
occurs  during  an  iteration,  then  both  P^  and  N|  = 0 for  some  i e IR.  The 
sum  of  the  absolute  values 


n 

S = z 


■■■  ■ Vml  ' 'E<^R  - V> 

■ ®^*^R  ■ ’‘r*B*  ''B^ 


i 


We  shall  now  assume  that  degeneracy  is  present  during  an  iteration. 

We  may  note  that  ^5  “ ^ during  the  iteration  because  they  are  nonbasic. 

For  simplicity,  we  shall  assume  that  ^8(1)  enter  the  btsis.  If  we 

slightly  increase  Pq/.x,  then  we  can  use  this  to  determine  the  condition  for 

Bp ; 

a decrease  in  the  sum  of  absolute  values.  We  take  PgCfi)  = (0,  ...,  0,  6,  0,  ....  0) 
where  6 > 0 and  6 is  the  ith  component  of  Pj.  Our  equations  will  now  take  the 

D 

form: 

» = ' "b'^'b  - *80 )' 

E(P|,  - «„)  - E(y„  - X„x:V)  + EXoX;'p„(«) 


= E(yR  - Xj^Xg  yg)  + EXj^Xg^^jS 


Let  ID  be  the  index  set  of  the  degenerate  variables  and  IR  = IR  - ID.  We 
can  now  rewrite  the  above  in  a partitioned  vector  notation  as: 


/iR*  ''''  ■ /iR*  ‘"'b  ■ 

E(Br  - Nr)  ' / 

I E (P„  - N„)  / 1 E X|,x;J,)  « 


R R' 


The  new  sum  of  the  absolute  values  is 


s(e)  . 6 . ej(y,  - XjX-\)  * ejXjX-J.)  . . . ^Jx^x’l,,  .| 

= S + 6(1  + Z ^ ejXjX'L)  + t 1 X.X")  6| 

jelR  ''  ^ jelD 


Thus  to  have  the  condition  that  S(6)  < S,  we  must  have 
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Similarly,  if  N-..»  were  to  be  chosen  to  be  increased,  the  criterion  would 

Dll ; 

be 


jelR 


jelD 


'‘j''B(i) 


(II) 


By  using  this  new  criterion  for  basis  entr7,  the  minimum  ratio  rule  is 
thus  modified  to  the  smallest  positive  ratio  to  determine  the  vector  to  leave 
the  basis.  The  procedure  for  performing  the  multiple  pivots  will  remain  the 
same.  There  will  no  longer  be  any  degenerate  pivots,  and  the  objective  function 
will  strictly  decrease  at  each  iteration.  However,  examples  can  be  constructed 
where  use  of  this  criterion  alone  will  indicate  that  a basic  edge  with  a negative 
rate  of  change  does  not  exist,  yet  the  corresponding  extreme  point  is  not  optimal. 
Thus  the  standard  LP  criterion  should  be  used  to  verify  optimality. 


5.  Examples 


In  the  first  example,  we  shall  illustrate  the  difficulty  caused  by 
degeneracy  when  using  the  standard  simplex  criteria  for  choosing  the  vector 
to  enter  the  basis.  We  shall  then  present  the  sequence  of  tableaus  when 
the  new  criteria  for  selection  is  applied.  The  multiple  pivots  will  be 
denoted  by  asteriks.  The  simplex  format  will  consider  LP(2)  written  in  equiva- 
lent maximization  form,  and  it  will  follow  a combination  of  the  notation  of 
Davies,  and  Barrodale  and  Roberts.  For  brevity,  the  revised  simplex  tableaus 
will  be  used.  The  problem  is  to  find  the  LAV  estimators  for  the  model 
y = + BgX  from  the  following  data: 

X -5  -3  -3  -2  0 1 2 2 3 4 4 

y 0 2 -3  -1  0 1 3 -1  2 0 4 

The  initial  basis  matrix  Xg  will  consist  of  the  sixth  and  eleventh  observations, 
and  degeneracy  will  occur  in  this  initial  tableau.  The  rule  which  we  used  to 
determine  the  assignment  of  or  P^.  as  the  degenerate  basic  variable  in  this 
initial  tableau  was  to  use  the  sign  of  the  y^  from  the  data,  i.  e.,  since  y^  = -3, 
we  assigned  Ng  as  the  degenerate  basic  variable,  but  for  y^  = 0,  we  arbitrarily 
assigned  Pg.  Some  algorithms  use  the  assignment  of  a P^.  for  each  degenerate 
value  that  occurs.  This  rule  can  also  lead  to  computational  problems.  The 
sequences  of  pivots  are  quite  easy  to  follow  when  a graph  of  the  parameter  (Bj,  Bg) 
space  is  used.  Finally,  the  optimal  tableaus  also  indicate  that  alternate  optimal 
solutions  are  available. 


BASIS 

TABLEAU  I 

RESIDUAL  Ng 

Pll 

BASIS 

TABLEAU 

RESIDUAL 

II 

P4 

Pll 

Pi 

5 

3 

2 

Pi 

21/6 

9/6 

3/6 

P2 

5 

7/3 

4/3 

P2 

23/6 

7/6 

1/6 

0 

-7/3 

-4/3 

S 

7/6 

-7/6 

-1/6 

P4 

1 

2** 

1 

Ne 

3/6 

-3/6 

3/6* 

P5 

0 

4/3* 

1/3 

^5 

4/6 

-4/6 

2/6 

^1 

0 

-4/3 

-1/3 

4/6 

-4/6 

2/6 

P7 

1 

2/3 

1/3 

P7 

4/6 

2/6 

-4/6 

Ns 

3 

-2/3 

1/3 

^8 

20/6 

-2/6 

4/6 

Ng 

1 

-1/3 

2/3 

Ng 

7/6 

-1/6 

5/6** 

Nio 

4 

0 

1 

•^10 

24/6 

0 

1 

^2 

1 

1/3 

1/3 

^2 

5/6 

1/6 

1/6 

20 

-5 

-4 

18  5/6 

5/6 

-13/6 

I 


. 


' 

BASIS 

TABLEAU 

RESIDUAL 

III 

"4 

P9 

BASIS 

TABLEAU 

RESIDUAL 

IV  - OPTIMAL 

P6  P9 

{ 

14/5 

8/5 

3/5 

Pi 

2 

-4 

3 

h 

18/5 

6/5 

1/5 

P2 

3 

-3 

2 

h 

7/5 

-6/5 

-1/5 

N3 

2 

3 

-2 

P6 

1/5 

2/5* 

-3/5 

N4 

1/2 

5/2 

-3/2 

Ng 

1/5 

-3/5 

2/5 

N5 

1/2 

3/2 

-1/2 

1/5 

3/5 

2/5 

'1 

1/2 

3/2 

-1/2 

: '’7 

8/5 

1/5 

-4/5 

P7 

3/2 

-1/2 

-1/2 

12/5 

-1/5 

4/5 

N8 

5/2 

1/2 

1/2 

*1* ' *• 

-r'v  p 

'vy^i  *^11 

7/5 

-1/5 

-6/5 

Pll 

3/2 

1/2 

-3/2 

■^10 

13/5 

1/5 

6/5 

^10 

5/2 

-1/2 

3/2 

^2 

3/5 

-1/5 

1/5 

^2 

1/2 

-1/2 

1/2 

16  1/5 

-2/5 

3/5 

16 

1 

0 

14 


I If  we  return  to  Tableau  I and  apply  our  new  criteria  (I)  and  (II)  to 

find  the  "nondegenerate"  rate  of  change  (of  decrease)  in  the  objective  function, 
we  shall  now  obtain  an  opportunity  cost  of  -7/3  for  Ng  and  -10/3  for  Pn. 

I 

I Upon  observing  Tableau  II,  we  find  that  the  objective  function  value  is 

j 20  - 7/3  • 1/2  = 18  5/6,  has  increased  from  0 to  1-7/3  • 1/2 | = 7/6,  and 

I Ng  increased  from  0 to  1-4/3  • 1/2|  = 4/6.  These  would  have  the  same  values 

[ (and  the  rest  of  the  tableau)  if  only  the  nondegenerate  pivot  had  been  per- 

formed. However,  since  Pjj  has  the  most  negative  reduced  cost,  then  its 
associated  edge  gives  the  greatest  rate  of  decrease  among  the  basic  vectors. 

The  (nondegenerate)  multiple  pivots  are  given  in  Tableau  lA.  The  following 
tableau  will  be  optimal  and  identical  to  Tableau  IV. 


TABLEAU  lA 


BASIS 

RESIDUAL 

^6 

f’ll 

^1 

5 

3 

2 

^2 

5 

7/3 

4/3 

'^3 

0 

-7/3 

-4/3 

* 

P4 

1 

2 

1 

P5 

0 

4/3 

1/3 

0 

-4/3 

-1/3 

P7 

1 

2/3 

-1/3 

^8 

3 

-2/3 

1/3 

Ng 

1 

-1/3 

2/3 

^10 

4 

0 

1 

1 

1/3 

1/3 

I 


20 


-7/3 


-10/3 


15 


Modifying  the  original  data,  we  shall  illustrate  the  possibility  of 
obtaining  a decrease  along  the  basic  edges.  We  require  an  additional 
observation,  the  twelfth,  of  x = 1/2,  y = 0 for  our  data.  The  initial 
matrix  Xg  will  consist  of  the  tenth  and  twelfth  observations,  and  degeneracy 
will  occur  in  Tableau  V.  This  is  not  the  final  tableau  as  the  standard  re- 
duced costs  of  the  simplex  algorithm  are  not  positive.  Thus  a degenerate 
pivot  is  performed,  and  using  the  perturbation  technique  of  Chames  (1952), 
we  find  that  Pg  and  Nj2  will  be  exchanged  in  Xg.  For  this  particular  example, 
the  replacement  of  the  twelfth  observation  by  either  the  first  or  the  fifth 
observation  in  Xg  will  give  a basic  edge  along  which  the  objective  function 
will  decrease.  It  should  also  be  noted  that  only  a single  degenerate  pivot 
will  be  executed  since  the  addition  of  twice  the  component  of  the  tableau 
of  either  of  the  tied  minimum  ratios  to  the  reduced  cost  will  make  it  positive. 

At  this  point,  it  will  be  very  beneficial  to  examine  our  criteria  which 
gives  the  actual  rate  of  change  of  the  objective  function  along  the  basic 
edges.  The  rates  of  changes  for  Pjq,  Pj2.  and  Nj2  ^re,  respectively  37/7, 

1/7,  29/7,  and  37/7.  Since  these  values  are  all  positive,  then  proceeding 
in  any  direction  along  these  basic  edges  would  increase  the  sum  of  the  absolute 
values.  To  demonstrate  this,  suppose  we  select  N^q  to  be  increased.  The  first 
minimum  ratio  is  associated  with  N^,  and  it  is  7/5.  Xg  will  consist  of  the 
fourth  and  twelfth  observations  when  only  this  pivot  is  accomplished.  The  point 
will  be  gj  = -1/5,  ^2  - 2/5,  and  the  sum  of  the  absolute  values  will  be  17  1/5. 

We  can  determine  the  change  in  the  objective  function  by  using  the  information 
provided  by  the  algorithm.  Thus,  we  have  17  1/5  = 17  + (1/7)  (7/5). 

We  shall  conclude  our  discussion  of  this  example  by  stating  that  optimality 
will  be  reached  in  another  two  tableaus  after  the  interchange  of  Pg  and  Nj2  in  Xg. 
The  optimal  solution  is  gj  = 1/5,  g2  = 3/5,  and  the  sum  of  the  absolute  values  is 
16  7/10.  Xg  will  consist  of  the  fourth  and  nineth  observations. 
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Modifying  this  example,  we  shall  illustrate  the  possibility  of  not 
obtaining  a decrease  along  the  initial  basic  edges.  We  require  an  additional 
observation,  the  twelfth,  of  X = 1/2,  y = 0 for  our  data.  The  initial  matrix 
Xg  will  consist  of  the  tenth  and  twelfth  observations,  and  degeneracy  also 
occurs  in  this  tableau  (V).  It  should  be  clear  that  the  tableau  is  not  optimal 
and  that  the  addition  of  twice  the  component  of  the  tableau  of  either  of  the  tied 
minimum  ratioes  to  the  reduced  cost  will  make  it  positive.  For  this  particular 
example,  the  replacement  of  the  twelfth  observation  by  either  the  first  or  fifth 
observation  in  Xg  will  give  a basic  edge  along  which  the  objective  function 
will  decrease.  The  optimal  solution  is  6^  = 1/5,  Bg  = 3/5,  and  Xg  consists  of 
the  fourth  and  nineth  observations. 


TABLEAU  V 


BASIS 

RESIDUAL 

PlO 

Ni2 

Pi 

0 

11/7 

18/7 

Pz 

2 

7/7 

14/7 

^3 

3 

-7/7 

-14/7 

N4 

1 

-5/7 

-12/7 

P5 

0 

1/7 

8/7 

Pe 

1 

-1/7 

6/7 

P7 

3 

-3/7 

4/7 

^8 

1 

3/7 

-4/7 

P9 

2 

-5/7 

2/7 

h 

0 

-1/7 

-8/7 

Pll 

4 

-7/7 

0 

62 

0 

2/7 

2/7 

17 

13/7 

-15/7 

i 
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6.  Degeneracy  and  Optimality 

Although  we  have  taken  advantage  of  the  special  structure  of 
the  linear  progranming  formulation  and  its  associated  geometrical  considera- 
tions to  find  a condition  that  should  improve  convergence  when  degeneracy 
occurs  during  an  iteration,  we  must  emphasize  the  fact  that  our  criteria 
(I)  and  (II),  in  general,  cannot  be  used  as  sufficient  conditions  for 
optimality.  The  last  example  in  the  preceding  section  clearly  demonstrates 
that  the  positivity  of  (I)  and  (II)  does  not  imply  optimality.  However,  if 
the  objective  function  has  assumed  its  minimum  value,  then  the  criteria  (I) 
and  (II)  will  be  nonnegative,  i.e.,  the  conditions  are  necessary.  Supple- 
mental conditions  will  now  be  developed  which  are  sufficient. 

Conditions  (I)  and  (II)  may  be  viewed  as  being  derived  from  a 
dynamic  perturbation  of  the  problem.  This  perturbation  forces  for  j e ID, 
either  Nj  or  Pj  into  the  basis  depending  on  whether  the  sign  of  is 

positive  or  negative.  Hence,  movement  along  a basic  edge  will  always  be 
"away  from"  other  nonbasic  hyperplanes  passing  through  the  point  3*.  To 
some  extent  this  dynamic  perturbation  is  theoretically  justifiable  because 
the  perturbation  as  defined  by  Chames  (1952)  depends  only  on  the  constraint 
index  and  this  ordering  is  arbitrary.  After  a nondegenerate  pivot  the 
perturbation  may  be  redefined  without  the  danger  of  cycling.  The  difficulty 
with  (I)  and  (II)  is  that  a different  perturbation  may  be  enforced  at  a 
single  iteration.  This  means  that  for  each  j,  with  Xj  a row  of  Xg,  we  ma^y  be 
looking  at  the  reduced  costs  for  different  problems.  If  there  are  p degenera- 
cies at  an  iteration  then  without  varying  Xg  there  are  2^  different  bases  of 
the  form  (1)  for  this  extreme  point.  A general  representation  of  the  reduced 
costs  for  these  2"’  bases  is: 
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1 + 


E 

jelR* 


e^X.XrJ  , 

J J B(i) 


E 

jelD 


1 


E 

jelR* 


E 

jelD 


(III) 


(IV) 


for  P|^  and  Nj^,  respectively,  where  X|^  is  the  i-th  row  of  Xg  and  d^  = +1  or  -1. 

The  current  solution  is  optimal  when  a set  of  d.  jelD  can  be  found 
such  that  (III)  and  (IV)  are  nonnegative  for  i = 1,  2,  ...m.  Maximizing  the 
minimum  value  of  (III)  and  (IV)  over  i = 1,  2,  ...m. is  an  integer  programming 
problem.  It  would  not  appear  to  be  computationally  beneficial  to  solve  this 
problem;  however,  certain  dj  are  readily  available  from  the  computation  of 
(I)  and  (II).  These  can  be  conveniently  used  to  verify  optimality  in  certain 
instances. 

7.  Remarks  on  Computation  Results 

We  modified  a multiple  pivoting  code  developed  by  Armstrong  and 
Frome  (1976)  to  include  our  criteria  for  finding  the  basic  edge  of  steepest 
descent.  The  algorithm  was  coded  in  Fortran,  and  the  criteria  were  imple- 
mented in  a straightforward  manner.  Our  objectives  were  to  determine  if  the 
number  of  pivots  and  the  execution  time  would  be  decreased.  We  chose  to  com- 
pare the  results  of  our  coded  algorithm  with  the  original  code  of  Armstrong 
and  Frome.  We  generated  our  data  from  discrete  distributions  since,  theoreti- 
cally, degeneracy  will  not  occur  in  the  case  of  continuous  distributions.  In 
most  of  the  problems  in  which  there  were  only  a few  (4  or  less)  degenerate 
variables  at  an  iteration,  our  code  required  slightly  less  pivots  than  the 
original  code,  but  our  execution  time  exceeded  theirs.  In  most  problems 
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with  several  (or  many)  degenerate  variables  at  an  iteration,  we  found  our 
algorithm  showed  a significant  improvement  in  the  number  of  iterations  so 
compared  to  the  other  code.  However,  in  terms  of  execution  time,  our 
algorithm  showed  only  a slight  improvement  in  speed  of  convergence  in 
some  problems,  and  in  other  problems,  it  was  slower  in  reaching  optimality. 

8.  Observations 

The  purpose  of  this  paper  was  to  illustrate  that  degeneracy  in 
LAV  estimation  can  increase  the  number  of  iterations  and  also  solution  time 
in  algorithms  which  have  been  shown  to  be  very  efficient.  In  problems 
where  a large  number  of  observations  are  taken  to  estimate  a few  parameters, 
it  is  very  likely  that  degeneracy  will  occur.  We  have  been  able  to  develop 
an  improvement  in  the  multiple  pivoting  algorithms  when  degeneracy  occurs 
which  enables  the  pivots  to  proceed  along  the  basic  edge  which  gives  the 
greatest  rate  of  decrease  in  the  objective  function.  The  results  of  our 
computational  study  verify  our  hypothesis  on  decreasing  the  number  of 
iterations.  The  current  direction  of  study  involves  a more  sophisticated 
implementation  of  the  criteria  into  the  code.  One  possible  direction  in 
which  an  extension  of  the  procedure  would  be  desirable  is  to  have  an 
efficient  method  of  determining  the  rate  of  decrease  for  all  edges  at  a 
point  where  degeneracy  is  present.  However,  to  pivot  through  all  possible 
bases  by  the  simplex  algorithm  to  find  the  rates  would  require  c{nH-k,m) 
pivots,  and  this  would  not  be  advisable. 
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Efficient  algorithms  have  been  developed  recently  which  utilize 
the  specialized  structure  of  the  linear  programming  formulation  for  the  problem 
of  least  absolute  value  estimation.  These  algorithms  generally  proceed  in  the 
direction  of  steepest  descent  along  an  edge  of  a convex  polyhedral  surface. 
However,  we  wilj  show  that  the  extreme  point  path  of  steepest  descent  may  not 
be  taken  when  degeneracy  occurs.  We  will  also  present  a criterion  that  deter- 
mines the  basic  edge  for  steepest  descent. 
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